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ABSTRACT 

A study was conducted to determine the educational 
objectives that the school districts in New Mexico consider most 
important. Secondary purposes were to determine whether the two 
methods yielded similar or different priorities and whether there 
were any systematic differences in the views of certain districts 
and/or kinds of raters. Twenty-seven school districts formulated 
important objectives in social studies, communication skills, math, 
and science. Each of the 153 objectives was printed on a card, and 
the cards were divided into 4 decks for each of the subject areas. 
Within each of 31 additional district, four teams of raters were 
formed, each consisting of a student, a teacher, an administrator, 
and a community representative. Each team member, sorted each deck of 
cards into three piles — below average importance, average importance, 
and above average importance — with a minimum of five in each pile. 
Each team also researched a consensus as to the 5-15 objectives it 
considered most important. The procedure was then repeated with the 
27 districts involved in developing the objectives. The results 
indicated that certain objectives tended to be considered much more 
important than others and that this trend was consistent across 
different kinds of raters and districts and the two methods (i.e., 
individual raters vs. group consensus). (KM) 
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An Evaluation of New Mexico's Educational Priorities 

Stephen P. Klein 
Educational Evaluation Associates* 

Purpose 

This study was conducted to determine the specific educational objec- 
tives that the school districts in the state of New Mexico considered to 
be the most important. Two secondary purposes were to determine whether 
the two methods used to ascertain those preferences yielded similar or 
different priorities and whether there were any systematic differences in 
the view of certain districts and/or kinds of raters. 

Procedures 

A representative random sample of 27 of the State's school districts <> 
were asked to develop what they considered to be important educational 
objectives in each of the following four areas: 

1. Social Studies 

2. Communication Skills 

3. Mathematics 

4. Science 

This construction process involved State Department of Education staff 
working with community representatives, students, and school and district 
personnel in each of the 27 districts sampled. 

The total pool of objectives from all districts were then edited and 
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additional ones written to ensure a comprehensive set in each of the four 
areas* A total of 153 objectives were constructed as well as at least one 
sample item for each objective to illustrate how it might be measured. 
Each objective and its sample item(s) were printed on a 5x8" card. In 
this way, a deck of cards was prepared for each of the four areas noted 
above . 

These decks of cards were taken to 31 additional high school dis- 
tricts by State Department of Education staff. Within each district, there 
were four teams of raters. Each team was composed of a representative from 
each of the following four types of raters: 

1. Students 

2. Teachers 

3. Administrators (e.g., principal, assistant 
superintendent) 

4. Community representatives (e.g., parent, 
school board member) 

Each team member was given a deck of cards for a given content area 

and was asked to sort them into the following three piles: 

— 1. Below Average Importance 

— 2. Average Importance 

— 3. Above Average Importance 

Each rater had to put at least five objectives into each pile, but beyond 
that there were no restrictions as to the number of objectives that had 
to be placed in any one pile. After a team member completed his sorting 
of the objectives and tallied his results, he then discussed with his other 
team members the reasons for his choices. Generally quick consensus was 
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then reached as to the 5-15 objectives that the team considered to be most 
important. This process took about one hour to complete including an ex- 
planation of how the objectives were developed and the rationale for the 
procedures. 

Once this step was completed, the teams switched decks of cards and 
repeated the process of making both individual ratings of the relative 
importance of each objective and achieving consensus as to the 5-15 ob- 
jectives within each set that they felt were the most important. This pro- 
cedure was repeated two more times so that each rater rated each goal area. 
The final step involved each team submitting its set of important objec- 
tives to the superintendent who in turn met with the team captains to de- 
termine the district 1 s consensus as to the 5-15 most important objectives 
within each of the four areas. Thus, the total time involved in the entire 
process of selecting objectives was about 5 hours per rater per district. 

A description of these procedures and the decks of cards were then 
mailed to the 27 districts that were involved initially in the development 
of the objectives. These districts carried out these procedures with re- 
latively little help from State Department staff. 

All the foregoing procedures produced the following two sets of data: 

1, Each rater's individual judgment as to the rela- 
tive importance of each of the 153 objectives 
(the total number of judgments equalled 58 
districts X 16 raters X 153 objsctives). 

2. The consensus judgment within each of the 58 
districts as to the 5-15 objectives that it 
felt were the most important in each of the 
four areas surveyed. 
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Resultg 

Tables 1-4 contain the results of the analyses of variance for each 
area for the first type of data collected; i.e., the independent indivi- 
dual ratings of each objective. An inspection of these tables indicates 
almost identical results in each area. These results may be summarized 
as follows: 

1. The main effects due to the District and Rater Type factors were 
statistically significant at .001. This means that certain districts and 
certain rater types tended to be more lenient than others in their assign- 
ment of objectives to the three categories of importance; e.g., putting 
nore or less objectives into the category of "an important" objective. 

This result suggests that the procedural requirement of putting at 
least five objectives into each of the three categories of importance did 
not substantially curtail the raters 1 ability to indicate the nature of 
their preferences. 

2. The main effect due to the objectives factor was statistically 
significant at .001. This result was very important since it indicated 
that some objectives were consistently rated much more important than oth- 
ers. Thus, the observed differences between the mean ratings of objectives 
within an area are interpretable and not just due to chance fluctuations. 

3. None of the interactions were practically significant although 
the districts by objectives effect appeared to reach the level of statis- 
tical significance because of the very large number of ratings gathered. 
This is a very interesting result because it means that one type of rater 
and/or district did not tend to rate the objectives substantially differ- 
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ently (in the sense of selecting certain ones as being more or less impor- 
tant than others) than did other types of raters and/or districts. This 
finding was very important because it means that one can use the over- 
all rating for a given objective as indicative of what New Mexicans think 
about it without being overly concerned that a given rater type and/or 
district was not represented properly. On the other hand, issues dealing 
with particular districts should probably heed the obtained statistically 
significant result that some districts rated the objectives somewhat dif- 
ferently than did other districts. 

One of the major questions in the present study was whether the two 
methods for determining the relative priorities of objectives would lead 
to similar or different results. In other words, was the average individual 
rating for an objective indicative of whether the district finally selected 
it as being one of the 5-15 objectives they considered to be most important 
for a given area. The mean rating for each objective and the number of times 
it was selected by the 57 districts as being important has been tabulated in 
Appendix A. Table 5 contains the correlation coefficient between these two 
methods for each area. An inspection of these coefficients indicates that 
there is generally a very high but not a perfect agreement between the re- 
sults obtained with the two methods for determining priorities. 

Table 5 

Correlation between the average individual 
rating of each objective and the number of 
times it was chosen as being one of the 5-15 
most important objectives by a district 
Area Rated Correlation 

Social Studies .94 

Communication Skills .88 

Mathematics .96 

Science .88 
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The final set of analyses involved an examination of the intercor- 
relations among the objectives within each area* The purpose of this 
analysis was to determine whether there were any objectives that appeared 
to be duplicates. An inspection of the four intercorrelation matrices 
revealed no practically significant r's in the sense that certain objec- 
tives appeared to be rated in essentially the same manner as any other 
objective (no r was higher than .55 and most were below .10). 

Discussion 

The results of this study indicated that certain objectives tended to 
be considered much more important than others. Further, this trend was 
consistent across different kinds of raters and districts and the two methods 
employed for determining the relative importance of each objective (i.e., 
average across raters vs. group consensus). It appears, therefore, that 
there is widespread unanimity within the State as to the relative impor- 
tance of various educational objectives since a representative sample of 
two- thirds of the State's 89 districts participated in this study. 

There are, however, a number of limitations of the present study. 
First, the choice of who represented each type of rater within a district 
was determined by a member of the district' staff so it may be assumed that 
it was not always as random as the procedures dictated. The fact that the 
Raters X Districts interaction was not significant suggests that this was 
not an important bias. Second, a total of 75 sets of ratings were not ob- 
tained (i.e., abdut 10% missing data) and dummy data had to be inserted 
for them (the average racing for the objective for the type of rater who was 
missing) . This loss did not appear to be systematic as to the type of 
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rater or district and thus, it was assumed to be random. Third, and per- 
haps most important, the instructions to the raters only required them 
to judge the relative importance of each objective without attention to the 
levels or standards of performance required. Thus, some changes in priori- 
ties might have occurred if the objectives also contained a description of 
the level of performance required, e.g., some raters may have felt that only 
a minimal level of a certain objective was very important and that this 
accounted for its high ratings. The somewhat unusual choices as to the 
important objectives within the area of communication skills may have been 
caused by this problem (e.g., punctuation and library skills were among 
the highest rated objectives while reading comprehension and writing skills 
were near the middle and bottom of the list, respectively). 

Cone lus ion 

The results of this study indicate the relative importance of educa- 
tional objectives within the State, provided one takes into consideration 
the limitations noted above. The two procedures used to obtain these ratings 
yielded essen' ? ally identical results and thus, it appears that they reflect 
accurately the present priorities. 



